Requirement Checklist (GCP)

Software Requirements

Ensure the following command-line tools (CLIs) are available on the machine you install from:

• kubectl
• helm
• helmfile
• gcloud (Google Cloud CLI)

We don't pin any specific versions in the Yeedu installer. Use kubectl and helm releases that are compatible with the Kubernetes version of your cluster, and a helmfile release built against that helm version.

note

yeedu-core-services.sh installs kubectl, Helm, the Helmfile CLI, psql, rabbitmqadmin and the Redis CLI on the machine if they're missing, adapting to the machine's architecture and OS.

Cloud Requirements

1. Google Cloud Platform (GCP) project

Make sure to have an active GCP project.

2. Google Cloud Platform (GCP) Service Account

Create a service account with following permissions: Compute Engine Admin, VPC Reader, GCR Reader, Storage Object Admin, Kubernetes admin

3. Google Kubernetes Engine (GKE)

Configure Google Kubernetes Engine with the following recommendations:

• Namespace: yeedu (Recommended)
• Cluster Version: a Kubernetes version that is still in standard support on GKE
• Node capacity: at least 8 vCPU and 16 GB RAM available to the Yeedu control-plane workloads (the Yeedu CloudFormation template sizes its control-plane host at "Min 8 core, 16 GB RAM")

4. Google Cloud Storage (GCS)

Create a bucket in the same project with Storage Object Admin Access. Ensure the service account from Step 2 has the necessary read and write permissions.

5. Google Cloud SQL (PostgreSQL)

Deploy a Google Cloud SQL (PostgreSQL) instance with the following configuration:

• Engine: PostgreSQL 15.x
• Database Name: yeedu
• Backup Policy: Everyday (For Production)
• High Availability and DR: Recommended for Production

note

Yeedu doesn't publish a capacity model for the metadata database. The only version constraint pinned anywhere in the product is PostgreSQL >=15 and <16 (the Yeedu AWS CloudFormation template, automation/aws/yeedu-platform.yaml). Size CPU, memory and storage to your own workload.

6. Google Container Registry (GCR)

Create an Google Container Registry (GCR) for storing Docker images. Refer to setup-docker-registry section for a list of container registries to be created and instructions on uploading Yeedu images.

7. Google Stackdriver

Set up Google Cloud Logging to store logs. Ensure the service account from Step 2 has the necessary write access.

8. Firewalls

Ensure the following ports are open:

Service Name	Port	Access From Workstation	Access From Apache Spark Cluster	Access From Yeedu Control Plane Server
REST API	8080	Yes	Yes	Yes
Jupyter Notebook	8888-9088	No	No	Yes
RabbitMQ UI	15672	Yes	No	No
Redis	6379	No	No	Yes
RabbitMQ	5672	No	Yes	Yes
PostgreSQL	5432	Yes	Yes	Yes
History-server	10000	Yes	No	Yes
Grafana	3000	Yes	Yes	Yes
Influx DB	8086	No	Yes	Yes
LDAP	389	No	No	Yes

The following ports are also used by the platform. They're defined in bin/yeedu-env.sh:

Service Name	Port	Environment variable / notes
Yeedu UI	5173	YEEDU_UI_PORT
Vault	8200	YEEDU_RESTAPI_VAULT_PORT
Vault UI	9200	YEEDU_RESTAPI_VAULT_UI_PORT
Yeedu Functions scheduler	5171	YEEDU_FUNCTIONS_SCHEDULER_PORT
Yeedu Functions proxy	5172	YEEDU_FUNCTIONS_PROXY_PORT
Cache Proxy	8765	YEEDU_CACHE_PROXY_PORT; used only when YEEDU_CACHE_PROXY_ENABLED is true
Scheduler (Airflow)	8081	YEEDU_SCHEDULER_PORT; used only when YEEDU_ENABLE_SCHEDULER is true
AssistantX	8000 / 8001	YEEDU_ASSISTANT_PORT / YEEDU_ASSISTANT_FASTAPI_PORT; used only when YEEDU_ASSISTANT_ENABLED is true
Filestore (NFS)	2049	Required from the control-plane host to the Filestore instance

Make sure to fulfill these prerequisites for a successful Yeedu installation.

Google Cloud Filestore prerequisites

• Provision a Filestore instance within a Google Cloud project.
• Ensure the Filestore instance resides in the same VPC network and subnet as the connected resources.
• Access Control: Grant admin access to the listed ip ranges.